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SECTION  1 


INTRODUCTION 

This  is  the  Final  Technical  Report  for  the  Passive  Autonomous  Infrared  Sensor  Technology  Program,  Contract  No. 
DAAL02-85-C-0134.  The  report  covers  the  investigation  of  two  of  Honeywell’s  detection  and  segmentation 
algorithms  as  applied  to  automatic  target  recognizers  (ATRs).  The  contract  period  was  from  August  9,  1985  to 
May  9, 1987. 

1.1  BACKGROUND 

ATR  technology  is  entering  a  crii  .  phase  of  the  technology  cycle.  ATR  functions  are  now  being  incorporated  as 
a  requirement  in  several  defense  systems  like  the  Advanced  Attack  Helicopter  (AAH),  and  Light  Helicopter 
Experimental  (LHX).  These  systems  will  impose  increasingly  demanding  requirements  on  ATR  performance. 

During  the  course  of  ATR  evolution,  numerous  algorithms  have  been  developed  and  tested.  However,  the 
algorithms  are  limited  to  a  very  small  scope  of  applications.  Outside  this  limited  scenario,  many  ATRs  fail  their 
performance  requirements.  For  future  systems,  the  ATR  algorithms  should  be  able  to  maintain  optimal 
performance  under  a  variety  of  applications,  such  as  scenarios,  environments,  countermeasures,  clutter  density,  and 
others. 

For  these  reasons,  the  Defense  Advanced  Research  Projects  Agency  (DARPA)  has  initiated  an  effort  to  establish  an 
improved  procedure  with  standards  of  measurement  for  investigating  ATR  algorithm  performance. 

The  PAIRSTECH  Project  is  part  of  this  larger  DARPA  program  that  includes  active  infrared  and  MMW  sensors, 
but  as  the  name  implies,  the  PAIRSTECH  Project  focuses  on  image  processing  in  the  thermal  infrared,  initially  8  to 
12  Jim  for  tactical  air-to-ground  applications. 

1.2  PROBLEM  DESCRIPTION 

Two  broad  problems  are  addressed  in  the  overall  PAIRSTECH  Project:  1)  how  to  investigate  the  ATR  as  a  whole 
and  2)  how  to  evaluate  the  performance  of  ATRs  on  realistic  and  varied  data  to  determine  and  compare  both  the 
capabilities  and  limits  of  the  existing  ATR  systems.  The  first  problem  is  addressed  by  breaking  the  ATR  system 
into  separate  modules  and  investigating  each  module  individually.  The  modules  that  were  agreed  upon  by  the  ATR 


community  are  detection,  segmentation,  feature  extraction,  and  classification.  The  second  problem  was  dealt  w  ith 

by: 


•  Characterizing  images  in  terms  of  image  metrics-image  metrics  are  a  small  set  of  parameters  that  are 
designed  to  capture  the  variations  that  exist  among  different  images  due  to  variations  in  targets, 
background  sensor,  and  other  parameters. 

•  Developing  standard  performance  measure  definitions,  so  that  the  performance  of  different  ATR  systems 
can  be  compared 

•  Tabulating  the  performance  measures  of  each  module  in  terms  of  image  metrics 

13  PROGRAM  OBJECTIVES 

This  project  mainly  deals  with  the  evaluation  of  only  two  modules  from  one  of  Honeywell’s  ATR  systems:  Multi- 
Target  Acquisition  Processor  (MTAP).  It  is  to  be  noted  that  MTAP  is  an  integrated  system  that,  when  broken  into 
different  isolated  modules  like  BACORE  segmentation  and  detection,  may  produce  suboptimum  results. 

The  objectives  of  this  contract  were 

•  To  evaluate  Honeywell's  Background  Adaptive  Convexity  Operator  Region  Expander  (BACORE) 
segmentor  and  post-segment  Moving  Target  Detection  (MTD),  Multi-Target  Tracking  (MTT),  and 
Archival  Scene  Model  (ASM)  as  integral  parts  of  BACORE  multi-frame  processing  on  real,  ground- 
truthed  FLIR  data;  to  characterize  data  by  image  variables;  to  conduct  a  parametric  investigation  of  the 
operators;  to  develop  algorithm  models;  and  to  quantify  algorithm  performance  by  image  variables 

•  To  perform  all  of  the  necessary  work  to  modify  Honeywell's  Texture  Boundary  Locator  (TBL)  segmentor 
software  so  that  it  interfaces  and  integrates  with  the  BACORE  subsystem;  and  to  conduct  comparative 
studies  and  experiments  to  evaluate  the  two  segmentors  under  the  same  mission  conditions  (i.e.,  same 
image  data,  same  number  of  frames) 

Sections  2  through  6  address  in  detail  these  objectives.  Section  2  of  this  report  discusses  the  image  metrics,  the 
performance  measures,  and  the  data  bases  that  were  used  in  this  investigation. 

Section  3  describes  the  BACORE  investigation.  This  includes  both  single-frame  and  scqucnual-framc  experiments, 
and  performance  modeling. 
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Section  4  discusses  the  TBL  investigations. 

Section  5  presents  a  comparison  of  BACORE  single-frame  performance  against  the  TBL  performance  results. 
Section  6  presents  the  conclusion  and  a  summary  of  the  results. 

1.4  KEY  RESULTS 

As  mentioned  above,  the  objectives  of  the  PAIRSTECH  Project  were  data  characterization,  performance  evaluation 
for  BACORE  and  TBL,  modeling  of  BACORE,  and  comparison  of  TBL  and  BACORE  on  the  same  data  bases  and 
for  the  same  number  of  frames.  The  data  characterization  task  was  the  task  of  extracting  image  metrics  from  the 
imagery.  The  metrics  that  we  used  were  those  recommended  by  the  ATRWG  and  ERIM,  namely,  target 

interference  ratio  squared  (TIR^),  target-to-background  interference  ratio  squared  (TBIR^),  and  edge  strength  ratio 
(ESR).  The  performance  evaluation  measures  that  were  used  again  were  those  recommended  by  ATRWG  and 
ERIM,  namely,  probability  of  detection,  false  alarm  rate,  and  segmentation  accuracy.  Note  that  only  the  detection 
and  segmentation  modules  of  the  MTAP  were  funded.  The  measure  of  segmentation  accuracy  was  again  the  one 
that  was  recommended  by  ATRWG  and  ERIM,  namely,  the  normalized  areas  of  overlap  between  the  segmented 
and  the  ground  truthed  targets. 

The  performance  measures  were  obtained  for  detection  and  segmentation  modules  of  both  B  ACORE  single-frame 
and  multi-frame  processors  and  the  TBL  algorithm. 

For  each  frame  of  data,  a  set  of  performance  plots  as  functions  of  TIR^,  TBIR^,  ESR,  and  range  was  obtained. 
Moreover,  we  quantized  the  metric  space  uniformly  and  tabulated  the  statistics  of  the  performance  measures  for 
each  metric  bin. 

What  do  all  these  data  lead  to?  Using  the  available  data  from  multi-frame  BACORE,  we  generated  models  from 
performance  measures  as  functions  of  image  metrics  and  range.  A  refinement  of  these  models  can  be  used  for 
predicting  the  behavior  of  BACORE  in  metric  domains.  It  is  conceivable  that  similar  modeling  could  be  done  for 
TBL  and  single-frame  BACORE. 

As  pan  of  this  project,  we  obtained  insight  into  the  behavior  of  BACORE  algorithm  parameters.  The  number  of 
internal  parameters  that  were  sensitive  to  variations  in  imagery  was  found  to  be  one-half  of  the  original  set.  This 
will  be  useful  in  any  future  automatic  setting  of  the  parameters. 


For  the  TBL,  the  extensive  sensitivity  analysis  showed  quantitatively  that  the  performance  of  TBL  (segmentation 
accuracy)  can  be  effectively  optimized  by  choosing  a  proper  set  of  parameters.  However,  it  was  found  that  each 
target  in  the  image  has  its  own  optimum  set  of  parameters.  This  observation  will  be  useful  for  any  future  use  of 
TBL  or  for  its  improvement 

The  overall  performance  values  of  TBL  and  BACORE  single-frame  processing  were  not  impressive.  The  average 
segmentation  accuracy  was  found  to  be  around  0.35  out  of  a  maximum  of  1 .  This  can  be  justified  partly  by  the  way 
in  which  ground  truthing  was  done  by  ERIM,  and  partly  by  the  way  segmentation  accuracy  was  defined. 

The  role  of  MTAP  clutter  rejection  was  shown  to  be  dubious  at  best.  Even  though  this  module  reduced  FAR,  it  also 
diminished  Pp>  to  a  large  extent.  In  one  data  base  (29-palm),  it  reduced  Pp>  from  0.72  before  clutter  rejection  to 
0.02  after  clutter  rejection. 

The  BACORE  multi-frame  algorithm  was  found  to  be  much  better  than  single-frame  BACORE  and  TBL. 

However,  for  multi-frame  work,  a  different  data  base  was  used  and  ground  truthing  was  done  by  us.  In  almost 
every  category,  BACORE  algorithms  outperformed  TBL.  However,  for  single-frame  BACORE,  these  advantages 
were  very  small. 

False  alarm  rate  was  found  to  be  around  1  to  3  per  frame  for  both  multi-  and  single-frame  BACORE  after  the  clutter 
rejection  module.  However,  before  the  clutter  rejection,  the  value  of  FAR  was  very  high. 

In  conclusion,  PAIRSTECH  has  provided  valuable  information  about  the  behavior  of  BACORE  and  TBL  on 
relatively  large  and  varied  data  bases.  For  the  first  time,  we  tabulated  the  performance  measures  as  functions  of 
image  metrics.  Image  metrics  are  still  a  developing  concept,  and  it  remains  to  be  seen  what  could  be  a  good  set  of 
metrics  to  truly  characterize  varied  imagery.  As  a  result  of  funding  limitations,  we  were  not  able  to  deeply  analyze 
the  wealth  of  information  that  we  have  gathered.  It  is  crucial  that  the  experience  of  PAIRSTECH  be  used  in  future 
designs  of  algorithms  to  create  "better"  ATRs  or  in  the  modification  of  existing  algorithms. 
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SECTION  2 


IMAGE  METRICS,  PERFORMANCE  MEASURES,  AND  DATA  BASES 

2.1  IMAGE  METRICS 

The  target  metrics  that  we  used  were  target  interference  ratio  squared  (TIR2),  target  background  interference  ratio 
squared  (TBIR2),  and  edge  strength  ratio  (ESR)  as  defined  by  ERIM.  TIR2  is  defined  as: 
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where  N0  =  the  number  of  pixels  in  the  object 

xQj  =  the  intensity  level  of  the  i-th  pixel  in  the  object 

Njj  =  the  number  of  pixels  in  the  background 

xj,j  =  the  intensity  level  of  the  i-th  pixel  in  the  background 

The  background  used  in  this  computation  is  a  local  window  around  the  minimum  bounding  rectangle  of  the  target. 

TBIR2  is  defined  similarly: 
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Edge  strength  ratio  (ESR)  is  defined  as  the  filtered,  range-compensated  Sobel  operator  output  divided  by  local 
background  variance. 

2.2  PERFORMANCE  MEASURES 


For  BACORE  single-frame,  BACORE  multi-frame  and  TBL,  a  set  of  performance  measures  is  defined  that 
characterizes  the  desired  measures  in  these  algorithms.  These  are:  probability  of  detection,  false  alarm  rate  (FAR), 
and  segmentation  accuracy. 


Probability  of  detection  (Pd)  is  defined  as: 

Pp  =  Number  of  Ground  Truth  Targets  Detected  (3) 

Number  of  Ground  Truth  Targets 

False  alarm  rate  (FAR)  per  frame  performance  is  defined  as: 

FAR  Performance  =  Total  Number  of  Clutter  Regions  Declared  Target  (4) 

Total  Number  of  Frames 

Segmentation  accuracy  (SA)  is  defined  as: 

SA  =  Number  of  Pixels  of  fSegmented  Region  Ground  Truth  Region!  (5) 

Number  of  Pixels  of  (Ground  Truth)  U  (Segmented  Region) 

For  BACORE,  two  sets  of  Pp  and  FAR  can  be  defined;  one  set  that  is  calculated  at  an  early  stage  of  the  algorithm 
is  referred  to  as  region  of  interest  (ROI)  set,  and  the  second  set  that  is  obtained  at  the  output  of  segmentation  and 
clutter  rejection  stage  is  referred  to  as  post  segment  (PS)  set.  For  TBL,  since  there  is  no  ROI  stage,  only  PS  set  is 
obtained. 

23  DATABASES 

As  part  of  the  PAIRSTECH  program,  we  had  access  to  relatively  large  data  bases,  each  exhibiting  a  wide  spectrum 
of  targets  at  various  ranges,  clutter  conditions  and  viewing  angles.  The  data  bases  that  were  used  in  . our 
experiments  were  the  following: 

1.  29-Palm  Data  Base-This  data  base  consisted  of  two  different  sets  each  containing  32  frames.  There  were 
about  184  targets  in  this  data  base. 

2.  Texas  Instrument  (TI)  data  base-This  data  base  consisted  of  three  different  sets.  Two  of  them  contained  45 
frames  and  the  remaining  one  contained  46  frames.  There  was  a  total  of  378  targets  in  this  data  base. 

Both  of  these  data  bases  have  ground  truth  data  available  which  consisted  of  the  wireframe  model  of  the 
targets  at  their  appropriate  locations  on  separate  frames  for  every  frame. 

3.  Texas  Instrument  Test  Data  Base-This  data  base  consisted  of  1 36  frames  of  unground  truthed  data. 

4.  29-Palm  Test  Data  Base-This  data  base  consisted  of  64  frames  of  unground  truthed  data. 

The  last  two  sets  were  used  only  to  perform  detection  and  segmentation  test  experiments.  The  results  of  these  tests 
were  submitted  to  ERIM  for  their  evaluation  experiments. 


5.  Honeywell  MSN1712  Multi-Frame  Data  Base-This  data  base  consisted  of  six  sequences  of  multi-frame 
data.  Each  sequence  had  20  frames  each  and  six  targets  per  frame,  with  the  exception  of  sequence  5,  which 
had  five  targets  per  frame.  This  data  base  was  characterized  and  ground  truthed  manually  by  trained  experts 
in  our  laboratory. 


SECTION  3 


BACORE  INVESTIGATION 


Honeywell’s  BACORE  algorithm,  shown  schematically  in  Figure  3-1,  is  well  documented  in  various  reports 
elsewhere  (Reference  1).  However,  for  the  sake  of  completeness,  a  brief  description  of  BACORE  is  made  in  the 
next  section. 

3.1  A  BRIEF  DESCRIPTION  OF  BACORE 

Figure  3-2  shows  a  more  detailed  description  of  detection  and  segmentation  modules  of  BACORE,  which  are  part 
of  the  BRITT  statistical  segmentation  (BST)  module  of  the  Honeywell's  BRl  l  1/MTAP  simulation. 

The  BST  single-frame  processing  algorithm  accepts  as  input  an  image,  a  range  map  and  a  set  of  control  parameters, 
and  generates  as  output  a  set  of  feature  vectors  that  characterizes  the  bright  convex  segments  which  were  detected 
by  the  algorithm.  The  algorithm  is  composed  of  a  two-pass  segmenter  and  a  feature  extractor.  The  segmenter 
developed  in  BRITT  Phase  I  involves  three  operations:  background  adaption,  a  convexity  operator,  and  region 
extraction,  and  is  thus  known  as  the  BACORE  segmenter.  The  first  pass,  which  includes  background  adaption  and 
the  convexity  operator,  is  generally  referred  to  as  "detection."  Region  extraction,  the  second  pass,  is  generally 
referred  to  as  "segmentation."  Detection  identifies  rectangular  subwindows  of  the  full  image  that  enclose  convex 
bright  regions.  These  regions  are  identified  by  virtue  of  the  fact  that  they  exhibit  a  locally  focussed  gradient  within 
the  region  interior.  This  first  pass  produces  regions  of  interest  (ROl),  general  regions  of  an  image  upon  which  more 
analysis  is  performed  during  the  second  pass,  segmentation.  Segmentation  extracts  segments,  each  of  which  has  an 
exact  shape  and  outline,  that  correspond  to  the  regions  of  interest  located  during  detection. 

The  algorithm  first  performs  subwindow  selection  and  target  size  normalization,  which  can  substantially  reduce  the 
throughput  requirements  of  detection.  Subwindow  selection  chooses  a  selected  portion  of  the  field  of  view  for  ROI 
detection  on  a  rotating  basis.  During  target  size  normalization  the  subwindow  is  undcrsamplcd  (oversamplcd)  to 
maintain  approximately  the  same  target  size  in  pixels. 

The  next  step  is  to  detect  ROI  in  the  normalized  subwindow.  Locally-adaptive  background  subtraction  is 
performed,  which  diminishes  bland  areas  and  enhances  bright  areas.  (Note:  The  terms  background  adaption  and 
background  subtraction  are  used  interchangeably;  background  "adaption"  is  achieved  by  subtracting  out  the 
background).  This  is  followed  by  globally-adaptive  bright  and  edge  dctccuon  Bright  dctccuon  applies  a  fixed 
threshold  to  generate  a  bright  image,  and  edge  detection  applies  an  cight-dirccuon  compass  gradient  operator  to  the 
background-subtracted  image  and  thresholds  the  resulting  image  to  form  the  edge  image.  The  bright  and  edge 
images  arc  logically  ANDed  together  to  form  the  bright/cdgc  coincidence  image.  Then  the  convexity  operator 
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detects  convex  regions  by  looking  for  areas  where  local  gradients  are  focused  and  the  segment  size  is  comparable  to 
a  target  size.  Other  ROI  outside  the  subwindow  are  identified  as  stored  from  previous  frames  in  the  ASM.  These 
positions  are  seeded  into  the  ROI  image  output  from  detection,  thus  a  full  frame  with  all  ROI  is  obtained. 

The  objective  of  region  extraction  is  to  define  the  exact  shape  and  outline  of  the  segments  which  correspond  to  the 
located  ROI.  The  search  for  specific  segment  outlines  uses  a  histogram  approach  that  effectively  considers  all 
possible  intensity  thresholds  within  a  window  around  the  ROI,  and  chooses  a  "reasonable"  one  for  determining  the 
segment’s  shape  based  on  gradients  in  the  window.  A  minimum  bounding  rectangle  is  generated  for  each  of  the 
extracted  segments,  along  with  a  chain-coded  representation  of  the  segment  perimeter  and  a  binary  mask  of  the 
segment  silhouette. 

After  the  two-pass  segmentation  processing  has  finished,  a  set  of  statistical  features  that  characterize  each  of  the 
located  segments  is  computed.  This  is  accomplished  by  first  calculating  the  moments  through  third  order  of  both 
the  intensity  image  and  the  binary  image  of  the  segment.  The  raw  moments  are  then  centralized  and  normalized, 
and  finally  a  set  of  invariant  moments  is  calculated  that  is  independent  of  any  translation,  rotation,  or  scaling  of  the 
segment  within  the  image  plane. 

The  output  of  the  BST  is  next  fed  to  the  single-frame  clutter  rejection  module  (SRJ).  The  purpose  of  the  single- 
frame  clutter  rejection  (SRJ)  algorithm  is  to  eliminate  from  further  consideration  those  segments  deemed  to  be 
clutter,  by  either  a  piece-wise  linear  (PWL)  classifier  or  the  combination  of  a  PWL  classifier  and  a  Bayes  classifier. 
The  algorithm  classifies  a  segment  based  on  its  associated  feature  vector  in  a  feature  vector  file  (produced  by  the 
BRITT  statistics  algorithm  BST).  This  feature  vectors  consist  of  features  extracted  from  segments  segmented  in  the 
current  frame  only,  hence,  the  term  "single-frame."  The  segments  determined  to  be  clutters  arc  eliminated  from 
further  processing  by  not  passing  their  silhouettes  from  the  input  silhouette  file  (also  produced  by  BST)  to  the 
output  silhouette  file.  This  is  significant  because  the  multi-target  tracking  (MTT)  algorithm,  following  SRJ,  will 
only  track  those  segments  whose  silhouettes  are  found  in  output  silhouette  file. 

Figure  3-3  demonstrates  the  functional  flow  of  the  SRJ  package.  In  Figure  3-3  the  archival  scene  model  (ASM)  is 
used  to  store  the  current  frame  feature  vectors  and  silhouettes. 

3.2  PERFORMANCE  OF  SINGLE-FRAME  BACORE  ON  COMBINED  TI  AND  29-PALM  DATA  BASES 
3.2.1  SA  YSJTIR2 

Figure  3-4  shows  the  variations  of  SA  as  functions  of  TIR^.  In  the  first  bin  (4x10'6<T1r2<139),  sa  has  an  average 
value  of  0.37  and  standard  deviation  of  0.24.  In  the  second  bin  (139  <TIr2<277),  SA  has  an  average  value  of  0.34 
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Figure  34.  Segmentation  Accuracy  vs.  TIR2  (complete  data  sct)-Full  Range  of  TIR2 
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and  standard  deviation  of  0.10.  In  the  third  bin  (277<TIR^<416),  SA  has  an  average  value  of  0.59  and  standard 
deviations  of  0.1 1.  In  the  fourth  bin  (4I6<TIR2<554),  SA  has  an  average  value  of  0.61  and  standard  deviation  of 
0.05.  By  expanding  the  range  of  TIR^  between  0  and  150,  furthermore,  as  shown  in  Figure  3-5,  one  obtains  the 
following.  In  the  first  sub-bin  (0<TIR^<37.5),  SA  has  an  average  value  of  0.36  and  standard  deviation  of  0.24.  In 
the  second  sub-bin  (37.5<TIR^<75),  S  A  has  an  average  value  of  0.42  and  standard  deviation  of  0.26.  In  the  third 
sub-bin  (75<TIR2<1 13),  SA  has  a  average  value  of  0.42  and  standard  deviation  of  0.18.  In  the  fourth  sub-bin 
(1 13<TIR2<150),  S  A  has  an  average  value  of  0.32  and  standard  deviation  of  0.22. 

3.2.2  SA  vs.  TBIR2 

Figure  3-6  shows  the  variation  of  SA  as  a  function  of  TBIR^.  In  the  first  bin  (3xlO'^<TBIR^<25.5),  SA  has  an 
average  value  of  0.37  and  a  standard  deviation  of  0.24.  In  the  second  bin  (25.5cTBIR2<51.1),  SA  has  an  average 
value  of  0.45  and  a  standard  deviation  of  0.17.  In  the  third  bin  (51.1<TBIR^<76.6),  there  are  no  samples.  In  the 
fourth  bin  (76.6cTBIR2<102),  there  is  only  one.  sample  which  has  a  SA  value  of  0.25.  By  expanding  the  region 
between  0  and  25,  as  shown  in  Figure  3-7,  one  obtains  the  following  results  In  the  fust  sub-bin  (0<TBIR^<6.25), 
SA  has  an  average  value  of  0.30  and  standard  deviation  of  0.22.  In  the  second  sub-bin  (6.25<TBIR2<12.5),  SA  has 
an  average  value  of  0.51  and  standard  deviation  of  0.22.  In  the  third  sub-bin  (12.5<TBIR2<18.8),  SA  has  an 
average  value  of  0.55  and  standard  deviation  of  0.19.  Finally,  in  the  fourth  sub-bin  (18.8<TBIR^<25.0),  SA  has  an 
average  value  of  0.53  and  standard  deviauon  of  0.14. 

3  2.3  SA  YS.  ESR 

Figure  3-8  shows  the  variation  of  SA  as  a  function  of  ESR.  In  the  first  bin  (9.44xl0<ESR<1810),  SA  has  an 
average  value  of  0.37  and  standard  deviation  of  0.24.  In  the  second  bin  (1810<ESR<3620),  SA  has  an  average 
value  of  0.28  and  standard  deviauon  of  0.09.  In  the  third  bin  (3620<ESR<5420),  SA  has  an  average  value  of  0.51 
and  standard  deviation  of  0.05.  In  the  fourth  bin  (5420<ESR<7230),  SA  has  an  average  value  of  0.66  and  standard 
deviation  of  0.09.  By  expanding  the  ESR  region  between  0  to  1500,  furthermore,  as  shown  in  Figure  3-9,  one 
obtains  the  following  results.  In  the  first  sub-bin  (0<ESR<375),  SA  has  an  average  value  of  0.35  and  standard 
deviation  of  0.24.  In  the  second  sub  bin  (375<ESR<750),  SA  has  an  average  value  of  0.44  and  standard  deviation 
of  0  24.  In  the  third  sub-bin  (750<ESR<1 1.30),  SA  has  an  average  value  of  0.41  and  standard  deviation  of  0.22. 
Finally,  in  the  fourth  sub-bin  (1 130<ESR<1500),  SA  has  an  average  value  of  0.48  and  standard  deviation  of  0.21 . 
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Figure  3-6.  Segmentation  Accuracy  vs.  TBIR2  (complete  data  set)-Full  Range  of  TBIR2 


U.UUb+UU  p—mm - 1 - 1 - f 

9  44E-01  1.81E+03  3.62E+03  5.42E+03 

ESR 


7.23E+03 


Figure  3-8.  Segmcnlation  Accuracy  vs.  ESR  (complete  data  scl)-FuIl  Range  of  ESR 


Figure  3-10  shows  the  variation  of  SA  as  a  function  of  range.  S  A  attains  its  highest  value  of  1.0  at  the  range  value 
of  7710  and  then  drops  to  around  0.8  at  the  range  value  of  10100. 


3.2.5  Pp  (ROI)  vs.  TIR2 

Figure  3-11  shows  the  variation  of  Pp  (ROI)  as  a  function  of  TIR2  As  TIR 2  increases,  Pp  tends  to  increase  and 
attains  the  value  of  1  for  all  TIR 2  values  higher  than  100. 

3.2.6  Pp  (ROI)  vs.  TBIR2 

Figure  3-12  shows  the  variation  of  Pp  (ROI)  as  a  function  of  TBIR2.  As  TBIR2  increases,  Pp  tends  to  increase 
and  attains  the  value  of  1  for  all  TBIR2  values  larger  than  15. 

3.2.7  Pp  (ROI)  vs.  ESR 

Figure  3-13  shows  the  variations  of  Pp  (ROI)  as  a  function  of  ESR.  As  ESR  increases,  Pp  tends  to  increase  and 
attains  the  value  of  1  for  all  ESR  values  larger  than  1000. 

3.2.8  Pp  (ROI)  vs.  Range 

Figure  3-14  shows  variation  of  Pp  (ROI)  as  a  function  of  range.  As  range  increases,  average  Pp  tends  to  increase 
and  attains  value  of  1  at  range  values  of  4900  to  7700.  Then  it  tends  to  decrease. 

3.2.9  Pp  (post  segment)  vs.  TIR2 

Figure  3- 1 5  shows  variation  of  Pp  (post  segment)  as  a  function  of  TIR2.  As  TIR2  increases  Pp  tends  to  increase 
and  attains  value  of  I  for  all  TIR2  values  larger  than  130. 

3.2.10  Pp  (post  segment)  vs.  TBIR2 

Figure  3-16  shows  variation  of  Pp  (post  segment)  as  a  function  of  TBIR2.  As  TBIR2  increases  Pp  tends  to 
increase  and  attains  value  of  1  for  all  values  of  TBIR2  larger  than  20. 
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3.2.11  P[>  (post  segment)  vs.  ESR 


Figure  3-17  shows  variation  of  Pg>  (post  segment)  as  a  function  of  ESR.  As  ESR  increases  Pq  tends  to  increase  and 
attains  a  value  of  1  for  all  values  of  ESR  larger  than  1 000. 

3.2.12  Pq  (post  segment)  vs.  Range 


Figure  3-18  shows  variation  of  Pp>  (post  segment)  as  a  function  of  range.  Average  Pp>  tend  to  increase  a  range 
increases.  It  attains  a  value  of  1  at  range  values  betv.  n  4900  to  7700.  Then  it  tends  to  decrease. 

3.2.13  P[)  (post  segment  and  clutter  rejection)  vs.  TIR^ 

Figure  3-19  shows  variation  of  Pd  (post  segment  and  clutter  rejection)  as  a  function  of  TIR^.  As  TIR-  increases, 
Pj}  increases  and  attains  a  value  of  1  for  all  TIR^  greater  than  150. 

3.2.14  Pd  (post  segment  and  clutter  rejection)  vs.  TBIR^ 

Figure  3-20  shows  variation  of  Pq  (post  segment  and  clutter  rejection  as  a  function  of  TBIR^.  As  TBIR^  increases 
Pq  tends  to  increase  and  attains  a  value  of  1  for  all  values  of  TBIR^  larger  than  20. 

3.2.15  Pq  (post  segment  and  clutter  rejection)  vs.  ESR 


Figure  3-21  shows  variation  of  Pq  as  a  function  of  ESR  As  ESR  increases,  Pp  tends  to  increase  and  attains  a  value 
of  1  for  all  values  of  ESR  larger  than  1800. 

3.2.16  Pd  (post  segment  and  clutter  rejection)  vs.  Range 


Figure  3-22  shows  variation  of  Pq  as  a  function  of  range.  As  range  increases  average  Pp  tends  to  increase  and 
attains  its  highest  average  value  of  1  for  range  values  between  4900  and  7300  Then  it  decreases. 

3.3  PERFORMANCE  OF  BACORE  MULTI-FRAME  PROCESSING  ALGORITHM  VS.  METRICS 

We  processed  six  scqucnual  data  sets,  20  frames  each,  from  the  MSN  1712  Honeywell  data  base.  Figures  3-23  to  3- 

28  show  a  set  of  representative  frames  from  each  of  the  six  sequences.  The  image  metrics  T1R-,  TBIR-  and  ESR 
have  been  computed  on  every  frame.  The  ground  truthing  necessary  in  computing  these  metrics  was  performed  hv 
a  trained  operator. 


Clutter  Rejection  Pq  vs.  Range  in  Meters  (complc 


c  3-24.  Representative  Frame  from  Sequence  2  of  MSN1712  Data  Base 


Figure  3-26.  Representative  Frame  from  Sequence  4  of  MSN  17 12  Data  Base 


Figures  3-29  and  3-30  show  that  sequences  1  and  2  basically  span  the  same  metric  space  as  defined  by  TIR2  and 
ESR.  In  addition,  sequence  2  contains  36  targets  that  do  not  overlap  in  metric  space  with  those  in  the  sequence  1 . 
The  detection  rate  for  sequences  1  and  2  are  100%  and  83%,  respectively.  These  results  are  considerable 
improvements  over  single-frame  processing  because  of  the  tracking  module  included  in  the  multi-frame  algorithm. 
The  tracker  was  able  to  coast  a  target,  based  on  previous  frame  data,  even  though  in  the  current  frame,  the  target  in 
questions  has  a  low  contrast.  The  detection  for  sequence  2  was  not  as  good  as  sequence  1  because  several  targets 
were  too  small. 

Figures  3-31,  3-32,  and  3-33  show  segmentation  accuracy  (SA)  versus  the  three  metrics  TIR2,  TBIR2  and  ESR, 
respectively,  for  sequence  1.  No  detected  targets  had  less  than  45%  segmentation  accuracy,  while  some  were 

segmented  at  86%. 


Figures  3-34,  3-35,  and  3-36  show  segmentation  accuracy  versus  TIR2’  TBIR2,  and  ESR  for  sequence  2.  The  range 
of  SA  values  are  similar  to  those  of  the  previous  sequence.  The  range  of  TIR2  for  sequence  3  was  from  0  to  18.9. 
For  sequence  4,  the  range  of  TIR2  was  from  0  to  numbers  >100.  For  sequence  5,  this  range  was  from  0  to  29.6. 
And,  finally,  for  sequence  6,  the  range  of  TIR2  was  from  0  to  68.4,  with  most  of  the  values  of  TIR2  clustered 
around  0.  This  clustering  around  0  was  due  to  the  fact  that  the  value  of  background  variance  was  large  and  the 
difference  of  mean  intensities  of  target  and  background  was  small.  Figures  3-37,  3-40,  3-43,  and  3-46  show  the 
ranges  of  TIR2  and  the  corresponding  SA  values  for  sequences  3  through  6,  respectively. 

The  range  of  the  TBIR2,  as  can  be  seen  from  Figures  3-38,  3-41,  3-44,  and  3-47,  varied  from  0  to  8.97  in  sequence 
3  and  from  0  to  numbers  >100  for  sequence  4.  For  sequence  5,  TBIR2  varied  from  0  to  a  large  number  >100. 
However  the  target  corresponding  to  large  TBIR2  values  was  not  detected,  and  Figure  3-38  shows  the  results  of 
TBIR2<15.  And  finally,  for  sequence  6,  TBIR2  varied  from  0  to  numbers  >100.  In  the  above,  all  values  of  TBIR2 
and  TIR2  that  were  larger  or  equal  to  100  were  put  equal  to  100  to  reduce  the  dynamic  range  for  display  purposes. 

The  range  of  ESR,  as  can  be  seen  from  Figures  3-39,  3-42,  3-45,  and  3-48,  varied  from  0  to  2660  in  the  third 
sequence.  ESR  varied  from  13.4  to  4220  for  the  fourth  sequence.  It  varied  from  0  to  451  in  the  fifth  sequence.  The 
variation  of  ESR  was  from  0  to  2000  in  the  sixth  sequence. 

Figures  3-37  to  3-38  show  the  variations  of  the  segmentation  accuracy  measure  vs.  target  interference  rauo  squared 

(TIR2),  target  background  interference  ratio  squared  (TBIR2)  and  edge  strength  ratio  (ESR).  Segmentation 
accuracy  values  of  0  correspond  to  the  cases  when  the  ground  truth  and  segmented  targets  did  not  overlap. 
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Figure  3-32.  Segmentation  Accuracy  as  a  Function  of  TBIR2  (sequence  1 ) 
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Figure  3-33.  Segmentation  Accuracy  as  a  Function  of  ESR  (sequence  1) 


Figure  3-36.  Segmentation  Accuracy  as  a  Function  of  ESR  (sequence  2) 
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SEGMENTATION  ACCURACY  vs  TIR2  (6th  SEQUENCE) 


Figure  3-46.  Variations  of  Segmentation  Accuracy  as  TIR2  Varies  for  Sequence  6  of 
the  Honeywell  MSN1712  Data  Base 


SEGMENTATION  ACCURACY  vs  (6th  SEQUENCE) 


Figure  3-48.  Variations  of  Segmentation  Accuracy  as  ESR  Varies  for  Sequence  6  of 
the  Honeywell  MSN1712  Data  Base 


3-3-1  SA  vs.  TIR2 


Segmentation  accuracy  (SA)  generally  increased  as  TIR2  increased.  For  the  third  sequence,  the  mean  value  of  SA 
varied  from  0.60  in  the  first  bin  (0<TIR2<4.7)  to  0.78  for  the  last  bin  (14.2<TIR2<18.7).  For  sequence  4  of  the 
Honeywell  sequential  data  base,  SA  mean  values  increased  from  0.71  in  the  first  bin  (0<TIR2<25)  to  0.72  in  the 
third  bin  (50<TIR2<75),  and  then  decreased  to  0.67  for  the  last  bin  (75<TIR2<100).  In  sequence  5,  mean  value  of 
SA  varied  from  0.66  in  the  first  bin  (0<TIR2<7.41)  to  0.91  in  the  fourth  bin  (22.2<TTR2<29.6).  In  sequence  6  the 
mean  TIR2  value  increased  from  the  value  of  0.64  in  the  first  bin  (0<TIR2<15.1)  to  0.71  in  the  fourth  bin 
(45.3<TIR2<60.4). 

3.3.2  SA  vs.  TIUR2 

The  mean  segmentation  accuracy  (SA)  for  sequence  3  of  the  Honeywell  MSN1712  data  base  increased  from  0.60  in 

the  first  bin  (0<TBIR2<2.24)  to  0.81  in  the  fourth  bin  (6.72<TBIR2<8.97).  For  sequence  5,  the  mean  SA  value 
increased  from  0.67  in  the  first  bin  (0<TBIR2<3.7)  to  0.91  in  the  fourth  bin  (1 1.3<TBIR2<15).  For  sequence  6  the 
mean  SA  value  decreased  from  0.63  in  the  first  bin  (0<TBIR2<25)  to  0.59  for  the  fourth  bin  (75<TBIR2<100). 

In  general,  the  mean  SA  value  increased  as  TBIR2  increased.  However,  for  sequence  6,  this  was  not  the  case  and 
the  mean  SA  value  decreased  as  TBIR2  increased. 

3.3.3  SA  vs.  ESR 

The  mean  segmentation  accuracy  (SA)  for  sequence  3  of  the  Honeywell  MSN1712  data  base  was  0.63  for 
(0<ESR<662).  For  sequence  4  the  mean  SA  decreased  from  0.71  in  the  first  bin  (0<ESR<1070)  to  0.57  in  the 
fourth  bin  (3170<ESR<4220).  In  sequence  5,  the  mean  SA  increased  from  0.66  in  the  first  bin  (0<ESR<1 1 3)  to 
0.72  in  the  second  bin  (1 13<ESR<226),  then  dipped  down  to  0.67  in  the  third  bin  (226<ESR<338).  Finally,  the 
mean  SA  value  went  up  to  0.85  in  the  fourth  bin  (338<ESR<451).  In  sequence  6,  the  mean  S  A  decreased  from 
0.63  in  the  first  bin  (0<ESR<25)  to  0.59  in  the  fourth  bin  (75<ESR<100). 

3.3.4  Pp  vs.  FAR 


Figure  3-49  shows  the  variation  of  Pp  vs.  FAR.  The  numbers  inside  the  parentheses  reflect  the  number  of  samples 
that  contributed  to  the  derivations  of  the  (Pp,  FAR)  points.  As  can  be  seen,  a  large  number  of  samples  are  clustered 
around  FAR  <  1  a  Pp  >  0.65. 


Figure  3-49.  Variation  of  Pq  as  Function  of  FAR 


3.4  BACORE  MODELING 


Modeling  the  performance  of  BACORE  as  a  function  of  a  set  of  image  metrics  is  useful  in  many  ways.  First,  the 
models  can  be  used  to  predict  the  behavior  of  the  algorithms  in  areas  where  no  data  are  available,  and  second,  the 
models  can  be  used  to  modify  and  adapt  the  algorithm  parameters  to  influence  the  algorithm  behaviors  in  desired 
ways.  We  used  seven  different  models  to  model  the  performance  of  the  multi-frame  BACORE  as  a  function  of  a  set 
of  image  metrics.  The  performance  measures  that  were  used  were  the  following: 

•  Segmentation  accuracy  vs.  TIR2 

•  Segmentation  accuracy  vs.  TBIR2 

•  Segmentation  accuracy  vs.  ESR 

•  Segmentation  accuracy  vs.  range 

•  Probability  of  post  segment  target  detection  vs.  TIR2 

•  Probability  of  post  segment  target  detection  vs.  ESR 

•  Probability  of  post  segment  target  detection  vs.  range 

In  this  experiment,  "post  segment"  refers  to  the  output  of  the  clutter  rejection  module.  For  the  experiment,  we  used 
the  Honeywell  MSN  17 12  sequential  data  base.  The  First  three  sequences  were  used  for  modeling  and  the  last  three 
sequences  were  to  be  used  for  validating  the  models.  In  the  modeling  part,  metric  space  was  quantized  uniformally 
into  four  consecutive  bins.  An  attempt  was  made  to  select  the  same  number  of  samples  in  each  bin.  This  process 
was  done  for  each  of  the  sequences.  Then  polynomials  of  up  to  nine  degrees  were  Fit  into  these  selected  data.  The 
polynomial  degrees  were  chosen  in  such  a  way  as  to  produce  minimum  residual  errors. 

Figures  3-50  to  3-52  show  the  models  for  each  of  the  performance  measures  as  polynomial  functions  of  a  set  of 
metrics. 

For  Segmentation  accuracy  vs.  TIR2,  the  derived  model  is  expressed  as  the  following: 

SA  =  0.71  -2.81x10-3(T1R2)+3.67x10-4(TIR2)2-9.05x10-6(TIR2)3 

+7.47x10-8(TIR2)4+  1.81xI0-10(TIR2)5  (6) 

The  standard  deviation  of  the  error  was  8. 1 3 1  x  10'2.  Due  to  the  lack  of  sufFicicnt  data  for  large  T1R2  values,  the 
model  should  be  considered  only  for  ranges  of  TIR2<150. 

For  segmentation  accuracy  vs.  TBrR2,  the  derived  model  is  expressed  as  the  following: 


SA  =  0.56+3. 18xlO-2(TBIR2)  -  3.02  x  10'4  (TBIR2)2 


(7) 


vs. 


The  standard  deviation  of  the  error  was  9.48xl0'2.  Due  to  the  lack  of  sufficient  data  for  large  TBIR2  values,  the 
model  should  be  considered  only  for  ranges  of  TBIR2<  1 50. 


For  the  segmentation  accuracy  vs.  ESR,  the  derived  model  is  expressed  as  the  following: 


SA  =  0.40+4.72x10-3(ESR)-2.10x10-5(ESR)2  (8) 

+3.91x10'8(ESR)3  -  3.33x10' 1  !(ESR)4 
+  1.25xlO'14(ESR)5  -  1.65x10-18(ESR)6 

The  standard  deviation  of  the  error  was  7.62x1  O'2. 

This  model  shows  peculiar  behavior  around  ESR  values  from  1000  to  2000,  where  the  model  falls  off  and  again 
increases  sharply.  Due  to  the  lack  of  sufficient  data  in  regions  of  ESR>1000,  we  have  to  treat  the  behavior  of  the 
algorithm  model  for  ESR>1000  with  low  confidence.  However,  more  studies  need  to  be  performed  to  explain  this 
phenomenon  if  indeed  future  experimentations  are  to  confirm  the  presence  of  this  unexpected  behavior. 

For  segmentation  accuracy  vs.  range,  the  derived  model  is  expressed  as  the  following: 

SA  =  -3.91+1.63xl0'3(Range)  -  1.42xlO'7(Range)2  (9) 

The  standard  deviation  of  the  error  was  0.1006.  This  curve  is  displayed  in  Figure  3-53.  As  expected,  the  SA 
increases  with  range,  reaches  its  optimum  at  ranges  of  5000  to  6000  and  then  decreases. 

For  probability  of  detection  vs.  TIR2,  the  derived  model  is  expressed  in  the  following: 

PD=  O.SS+O^CITR^-g.bSxlO^CnR2)2  (10) 

+7.36x1 0'6(TIR2)3  -  1.71x10'8(TIR2)4 

The  standard  deviation  of  the  error  was  0.20032 15.  This  curve  is  displayed  in  Figure  3-54.  The  model  should  be 

considered  at  the  ranges  of  TIR2>150.  SA  increases  with  the  increase  in  TIR2  and  then  decreases  to  less  than  0.5 
and  again  increases.  The  cause  of  this  behavior  needs  to  be  explored  in  the  future. 

For  probability  of  detection  vs.  ESR,  the  derived  model  is  the  following: 


PD=  0.58+1.44x10'3(ESR)  -  1.29xlO'6(ESR)2 
+2.67x10'10(ESR)+3 


(ID 


The  standard  deviation  of  the  error  was  0. 16555466.  This  curve  is  displayed  in  Figure  3-55.  Again  the  Pq  value 
increases,  remains  high,  and  then  decreases  and  goes  up  again.  The  range  of  high  confidence  in  the  model  is  <2000 
for  ESR.  Causes  of  the  downward  behavior  of  Pq  for  ESR  values  of  >1000  need  to  be  explored. 

For  Pq  vs.  range,  the  following  expression  is  derived: 

PD  =  0.7  (12) 

The  standard  deviation  of  the  error  was  9.084x10"^.  This  curve  is  displayed  in  Figure  3-56.  As  can  be  seen,  Pq  is 
insensitive  to  range  values.  This  is  partly  due  to  the  fact  that  range  variations  for  the  sequential  data  were  small. 
Consequently,  the  only  data  available  were  at  relatively  the  same  ranges. 


SECTION  4 


TBL  INVESTIGATION 

4.1  A  BRIEF  DESCRIPTION  OF  TBL 

Honeywell's  texture  boundary  locator  (TBL)  is  a  texture  segmentation  algorithm.  TBL  uses  intensity,  mean  and 
variance  texture  measures  as  the  features  for  region  discrimination.  It  also  uses  spatial  gradient  information  in  the 
texture  domain  to  aid  in  the  distinction  between  ranges.  Figure  4-1  is  the  flow  diagram  of  the  TBL  algorithm.  The 
texture  attributes  of  each  pixel,  namely  the  n  x  n  window  mean  m  and  the  window  standard  deviation  s,  are 
computed.  Next,  a  texture  gradient  is  computed  using  a  window  operator  in  the  texture  space  as  follows:  A  2k  x  2k 
window  is  selected.  The  window  about  the  pixel  p  looks  like: 
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where  Xj  =  (mj,  s; ),  the  pair  determined  from  an  n  x  n  window  about  the  pixel  in  the  Xj  location  in  the  original 
image.  The  texture  measure  is  calculated  by: 

Tex  =  maxKmj  -  mj  +  4)2  +  (sj  -  Sj  +  4)2]1/2  (13) 

where  0<i<3.  This  measure,  a  metric  in  (m,s)  space,  computes  the  change  in  texture  at  a  distance  k  about  the  pixel 
P- 

Next,  the  histogram  of  the  gradient  image,  which  is  employed  as  a  global  information  measure,  is  used  as  a 
threshold  to  guide  prototype  selection.  Specifically,  the  threshold  is  set  to  select  a  certain  percentage  of  the  high 
intensity  gradient  points  such  that  a  sufficiently  large  number  of  pixels  are  included.  Then  edge  thinning  is  applied 
to  locate  the  prototypes.  Finally,  a  pixcl-to-labcl  mapping  and  connected  component  analysis  generates  the 
segmentation  by  labeling  all  pixels  inside  the  prototypes  as  potential  target  regions  and  those  outside  as  other 
regions. 

Three  parameters  (n,  k,  p)  are  needed  to  operate  this  algorithm. 


Input  Image 


Segmented  Image 
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Figure  4-1.  Target  Boundary  Locator  Scgmcntor 


4.2  TBL  SENSITIVITY  ANALYSIS  EXPERIMENT 


The  objective  of  this  experiment  is  to  investigate  the  variations  of  the  TBL  segmentation  accuracy  as  the  three 
parameters  are  varied.  The  outcome  of  this  investigation,  which  provides  insight  into  the  sensitivities  of  the 
segmentauon  accuracy  with  respect  to  the  (n,  k,  p)  set,  is  used  for  the  appropriate  selection  of  the  parameters  for  a 
particular  data  base. 

In  our  first  experiment  we  used  the  29-palm  data  base.  The  first  28  images  in  this  data  base  were  divided  into  two 
major  groups  based  on  the  similarities  of  the  member  of  each  group.  From  each  group  a  single  typical  image  was 
selected.  Then  a  3x3x4  factorial  design  was  performed,  and  a  total  of  36  runs  on  each  frame  were  made.  Figures 
4-2  and  4-3  show  the  selected  frames  for  this  study. 

The  parameters  n  and  k  were  varied  by  three  different  values.  The  parameter  p  varied  by  four  different  values  since 
past  experience  had  suggested  that  it  was  the  most  sensitive  of  the  three  parameters.  Figures  4-4, 4-5,  and  4-6  show 
the  variations  of  segmentation  accuracy  vs.  the  percent  parameter  p  for  three  different  values  of  n  and  k  for  one  of 
the  frames.  As  can  be  seen  from  Figure  4-4,  SA  achieved  a  maximum  at  p  =  0.25  for  n=4  and  k=6.  As  we 
increased  the  value  of  n  from  4  to  5,  the  maximum  SA  reached  its  peak  at  n=5,  k=7  and  p  =  0.35.  Moreover,  the 
value  of  maximum  increased  also  from  about  0.31  to  about  0.36.  Figure  4-5  shows  the  effect  of  increasing  n  to  6. 

In  this  case,  SA  achieved  its  highest  value  of  C.41  at  n=6,  k=6,  &  p=0.35.  Consequently,  the  best  result  was 
achieved  at  n=6,  k=6,  and  p=0.35. 

Figures  4-7, 4-8,  and  4-9  show  the  variations  of  S  A  for  the  second  frame  as  functions  of  n,  k,  and  p.  S  A  achieved 
its  highest  value  of  0.48  at  n=6,  k=7  or  8,  and  p=0.15  as  seen  from  Figure  4-9. 

Based  on  the  results  of  this  experiment,  an  average  appropriate  parameter  set  was  derived.  This  set  was  then  used 
for  the  28  frames  from  29-palm  data  base. 

4.3  TBL  PERFORMANCE  ON  29-PALM  DATABASE 

Twenty-eight  frames  from  the  29-palm  data  base  were  segmented  by  the  TBL,  and  the  segmentation  accuracy 
performance  measures  were  derived. 

Figures  4-10, 4-11,  and  4-12  show  the  variations  of  the  SA  values  vs.  TIR^,  TBIR^  and  ESR  metrics  for  the  29- 
palm  data  set. 
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Figure  4-4.  Variation  of  Mean  SA  as  Percent  Value  p  Varies  for  k=4  and  n=w=6. 
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Figure  4-5.  Variation  of  Mean  SA  as  Percent  Value  p  Varies  for  k=D=5  and  n=w=6,  7,  and  8 
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4J.1  SA  vs.  TIR2 

As  can  be  seen  from  Figure  4-10,  for  the  TIR^  values  in  the  first  bin  (0.00185<TTR^<8.70),  SA  has  a  mean  value  of 
0.30.  In  the  second  bin  (8.7<TTR2<0.7.4),  the  SA  average  was  0.43.  In  the  third  bin  (17.4<TIR2<26.1),  the  SA 
average  was  0.49.  Finally,  in  the  fourth  bin  (26.1<TIR2<34.0),  the  SA  average  was  0.01,  due  to  only  one  sample 
that  produced  this  SA  value. 

4.3.2  SA  vs.  TBIR2 

Figure  4-11  shows  the  variations  of  SA  vs.  TBIR 2  metrics.  In  the  first  bin  (0.00 14 1<TBIR2<  1.58),  the  SA  average 
was  0.29.  In  the  second  bin  (1.58<TBIR2<3.15),  the  SA  average  was  0.37.  In  the  third  bin  (3.15<TBIR2<4.73), 
the  S  A  average  was  at  0.63.  Finally,  of  the  fourth  bin  (4.73<TBIR2<6.3),  the  SA  average  was  0.31. 

4.3.3  SA  vs.  ESR 

Figure  4-12  >ws  the  variations  of  SA  vs.  ESR  metrics.  In  the  first  bin  (0.944<ESR<103),  the  SA  average  was 
0.30.  In  tb«  cond  bin  (103<ESR<205),  the  S A  average  was  0.41.  In  the  third  bin  (205<ESR<307),  the  SA 
average  was  0.30.  Finally,  in  the  fourth  bin  (307<ESR<409),  there  was  only  one  sample  due  to  one  target  that  was 
not  detected. 

In  the  above  analysis  we  have  excluded  the  samples  that  produced  SA  values  of  0  in  the  computation  of  average  S  A 
values. 

4.4  TEXAS  INSTRUMENT  DATA  BASE  RESULTS 

In  this  experiment,  a  total  of  136  frames  containing  372  targets  were  used.  There  were  three  different  sets  of  data  in 
the  TI  data  base,  which  we  refer  to  as  T3002  to  T3004.  In  the  following,  the  results  for  each  set  arc  reported 
separately. 

4.4.1  T3002  Set  Results 

Figure  4-13  shows  the  variations  of  SA  as  function  of  TIR2.  In  the  first  bin  (0<TIR2<37.5),  the  average 
segmentation  accuracy  was  0.33.  In  the  second  bin  (37.5<TIR2<75),  the  average  S  A  was  0.47.  In  the  third  bin 
(75<TIR2<1 13),  the  average  value  of  SA  was  0.49.  Finally,  in  the  fourth  bin,  the  average  SA  value  was  0  due  to 
the  one  target  that  was  not  detected. 


4.4.1.1  Variation  of  SA  vs.  TBIR^-Fieure  4-14  shows  the  variations  of  SA  as  a  function  of  TBIR^.  In  the  first 
bin  (1.43x10-3<TBIR2<7.17),  the  average  SA  value  was  0.33.  In  the  second  bin  (7.17<TBIR^<14.3).  the  average 
SA  value  was  0.29.  In  the  third  bin  (14.3<TBER^<21.5),  the  average  SA  value  was  0.69.  Finally,  in  the  fourth  bin 
(21.5<TBIR2<28.7),  the  average  SA  value  was  down  to  0.36. 

4.4. 1.2  Variation  of  S  A  vs.  F.SR-Figure  4-15  shows  the  variations  of  S  A  as  a  function  of  ESR.  In  the  first  bin 
(0<ESR<750),  the  average  SA  was  0.33.  In  the  second  bin  (750<ESR<1500),  the  average  SA  was  0.61.  In  the  third 
bin  (1500<ESR<2250),  the  average  SA  was  0.49  due  to  only  one  sample.  Finally,  in  the  fourth  bin 
(2250<ESR<3000),  the  average  value  of  SA  was  0  due  to  one  sample  that  was  not  detected. 

4.4.2  T3003  Set  Results 

4.4.2.1  Variation  of  S  A  vs.  TIR^-Figure  4-16  shows  the  variations  of  the  S A  as  a  function  of  TIR^.  in  the  first 
bin  (0<TIR2<32.5),  the  average  SA  value  was  0.36.  In  the  second  bin  (32.5<TIR2<65),  the  average  SA  value  was 
0.43.  In  the  third  bin  (65<TIR^<97.5),  the  average  value  of  SA  was  0.36.  In  the  fourth  bin  (97.5<TIR2<130),  the 
average  SA  value  was  0.6. 

4.4.2.2  Variation  of  SA  vs.  TBIR2-Figure  4-17  shows  the  variation  of  SA  as  a  function  of  TBIR^.  In  the  first  bin 
(3xl9'6<TBIR2525.5),  the  average  SA  value  was  0.36.  In  the  second  bin  (25.5<TBIR2<5l.l),  the  average  SA 

value  was  0.60.  In  the  third  and  fourth  bin  (51.I<TBIR2<I00),  there  was  only  one  sample  due  to  a  target  that  was 
not  detected. 

4.4.2.3  Variations  of  SA  vs.  ESR-Figure  4-18  shows  the  variation  of  SA  as  a  function  of  ESR.  In  the  first  bin 
(4.06<ESR<1770),  the  average  SA  value  was  0.36.  In  the  second  bin  (1770<ESR<3520),  the  average  SA  value  was 
0.33.  In  the  third  bin  (3520<ESRS5250),  the  average  SA  was  0.28  due  to  one  sample.  Finally,  in  the  fourth  bin 
(5250<ESR<7000),  the  average  S  A  value  was  0.52. 

By  expanding  the  first  bin  in  four  sub-bins,  one  obtains  the  following  results:  in  the  first  sub-bin  (0<ESR<475),  the 
average  SA  value  was  0.32.  In  the  second  sub-bin  (47<ESR<950),  the  average  SA  value  was  0.45.  In  the  third  sub¬ 
bin  (960<ESR<1430),  the  average  SA  value  was  0.56.  Finally,  in  the  fourth  bin  (1430<ESR<1900),  the  average  SA 
value  was  0.43. 


4.4.3. 1  Variation  of  SA  vs.  TIR^-Figurc  4-19  shows  the  variation  of  S A  as  a  function  of  TIR^.  In  the  first  bin 
(3.74x  10'3sTIR2^76.5),  the  average  SA  value  was  0.38.  In  the  second  bin  (76.5<TIR2<153),  the  average  SA  value 
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Figure  4-19.  Variation  of  Segmentation  Accuracy  as  a  Function  of  TIR2  for  the  T3004 
TI  Data  Base 


was  0.18.  In  the  third  bin  (153<TIR2<230),  the  average  SA  value  was  0.39  due  to  only  one  sample.  In  the  fourth 
bin  (230<TIR2<306),  only  one  sample  was  located,  which  was  due  to  one  target  that  was  not  detected  By 
expanding  the  first  two  bins  into  four  equal-size  bins,  one  obtains  the  following  results.  In  the  first  sub-bin 

(0<TIR2<37.5),  the  average  SA  was  0.33.  In  the  second  sub-bin  (37.5<TIR^<75.0),  the  average  SA  was  0.55  In 
the  third  sub-bin  (75.0<TIR2<1 13),  the  average  value  of  SA  was  0.33.  Finally,  in  the  fourth  sub-bin 
(1 13<TIR2<150),  the  average  SA  value  was  0.03. 

4.4.3.2  Variation  of  SA  vs.  TBIR^-Figure  4-20  shows  the  variation  of  segmentation  accuracy  as  a  function  of 
TBIR^  In  the  first  bin  (4.10x10"^<TBIR2<8.50),  the  average  SA  value  was  0.34.  In  the  second  bin 
(8.50<TBIr2<17.0),  the  average  SA  value  was  0.42.  In  the  third  bin  (17.0<TBIr2<25.5),  the  average  SA  value 
was  0.5 '.  In  the  fourth  bin  (25.5<TBIr2<34.0),  there  was  only  one  sample  due  to  one  target  that  was  not  detected. 

4.4. 3.3  Variation  of  SA  vs.  F.SR-Figure  4-21  shows  the  variation  of  S A  as  a  function  of  ESR.  In  the  first  bin 
(3.03<ESR<436),  the  average  SA  value  was  0.34.  In  the  second  bin  (436<ESR<860),  the  average  SA  value  was 
0.43.  In  the  third  bin  (860<ESR<1300),  the  average  value  of  SA  was  0.01  mainly  due  to  one  target. 


Figure  4-22  shows  the  variation  of  SA  as  a  function  of  range  for  the  T3002  data  set.  As  can  be  seen,  SA  attains  its 
highest  values  at  ranges  between  3000  to  5470  and  then  decreases.  The  highest  S  A  is  0.897. 


Figure  4-23  shows  the  variation  of  SA  vs.  range  for  the  T3003  data  set.  In  this  data  set,  SA  attains  its  highest  values 
at  ranges  between  3010  to  5350  and  then  decreases.  The  highest  value  of  SA  is  0.781. 


Figure  4-24  shows  the  variation  of  SA  vs.  range  for  the  T3004  data  set.  SA  attains  its  highest  value  of  0.825  at 
3230.  The  highest  SA  value  decreases  to  0.6  at  5510  and  then  increases  again  to  0.72  at  around  7000. 


Figure  4-25  shows  the  variation  of  SA  vs.  range  for  the  first  28  frames  from  29-palm  data  base.  There  are  three 
distinct  ranges:  800,  1210,  and  1770.  In  the  highest  SA  occurs  at  1770.  At  this  range,  the  highest  SA  attains  a 
value  of  0.672. 


4.5  PERFORMANCE  OF  TBL  ON  COMBINED  DATA  BASES  OF  TI  AND  29-PALM 


Figure  4-26  shows  the  variation  of  SA  as  a  function  of  T1R2.  In  the  first  bin  (4.0xl0‘<s<TIR^<139),  SA  has  an 
average  value  of  0.35  and  a  standard  deviation  of  0.25.  In  the  second  bin  (139<TIR~<277),  SA  has  an  average 
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Figure  4-21.  Variation  ol'  Segmentation  Accuracy  as  a  Function  of  ESR  for  the  T3004 
TI  Data  Base 
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Figure  4-22.  Variation  of  Segmentation  Accuracy  as  a  Function  of  Range  for  the  T3002 
Data  Base 
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Figure  4-23.  Variation  of  Segmentation  Accuracy  as  a  Function  of  Range  for  the  T3003 
TI  Data  Base 
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Figure  4-24.  Variation  of  Segmentation  Accuracy  as  a  Function  of  Range  for  the  T3004 
TI  Data  Base 


value  of  0.35  and  a  standard  deviation  of  0.18.  In  the  third  bin  (277<TIR2<416),  SA  has  an  average  value  of  0.40 
and  a  standard  deviation  of  0.09.  In  the  fourth  bin  (416<TIR2<554),  SA  has  an  average  value  of  0.60  and  a 
standard  deviation  of  0  due  to  the  only  sample  detected  in  this  bin 


Figure  4-27  shows  the  variation  of  SA  vs.  TIR2  for  (0<TER2<150)  only.  By  dividing  this  metric  space  into  four 
equal  and  consecutive  bins,  one  obtains  the  following  results.  In  the  first  sub-bin  (0<TIR2<37  5),  SA  has  an 
average  value  of  0.31  and  a  standard  deviation  of  0.24.  In  the  second  sub-bin  (37.5<TIR2<75),  SA  has  an  average 
of  0.49  and  a  standard  deviation  of  0.25  In  thud  sub-bin  (75<TIR2<1 13),  SA  has  an  average  value  of  0  44  and  a 
standard  deviation  of  0.21.  In  the  fourth  sub-bin  (1 13<TIR2<150),  SA  has  an  average  of  0.03  and  a  standard 
deviation  of  0.02  due  to  two  detected  targets. 

4.5.2  SA  VS.TB1R2 

Figure  4-28  shows  the  variation  of  SA  as  a  function  of  TBIR2.  In  the  first  bin  (3xlO~6<TBIR<25.5),  SA  has  an 
average  ''•due  of  0.35  and  a  standard  deviation  of  0.24.  In  the  second  bin  (25.5<TBIR2<51.5),  SA  has  an  average 
value  of  0.60  and  a  standard  deviation  of  0  due  to  the  presence  of  only  on  sample  in  this  bin.  In  the  third  bin 

(51.1<TBIR2<76.7),  no  sample  was  available  In  the  fourth  bin  (76  6<TBIR2<102),  there  was  one  target  that  was 
not  detected.  By  expanding  the  values  of  TBIR2  between  0  to  25,  as  shown  in  Figure  4-29,  the  following  results  are 
obtained.  In  the  first  sub-bin  (0<TBIR2<6.25),  the  SA  has  an  average  value  of  0.32  and  a  standard  deviation  of 
0.22.  In  the  second  sub-bin  (6.25<TBIR2<12.5),  SA  has  an  average  value  of  0.38  and  a  standard  deviation  of  0.28. 
In  the  third  sub-bin  (6.25<TBIR2<18.8),  SA  has  an  average  value  of  0.46  and  a  standard  deviation  of  0.28.  In  die 
fourth  sub-bin  ( 18.8<TBIR2<25),  SA  attains  an  average  value  of  0.45  and  a  standard  deviauon  of  0.08. 

4.5.3  SA  VS.  ESR 

Figure  4-30  shows  the  variation  of  SA  as  a  function  of  ESR.  In  the  first  bin  (0.944<ESR<1810),  SA  has  an  average 
value  of  0.35  and  a  standard  deviation  of  0.24.  In  the  second  bin  (1810<ESR<.3620),  SA  has  an  average  of  0.33  and 
a  standard  deviation  of  0  due  to  the  presence  of  a  single  sample.  In  the  third  bin  (3620<ESR<5420),  SA  has  an 
average  value  of  0.39  and  a  standard  deviation  of  0.1 1.  In  the  fourth  bin  (542(kESR<723()),  SA  attains  an  average 
value  of  0.52  and  a  standard  deviation  of  0.08.  By  expanding  the  ESR  range  between  0  to  1 5(X).  furthermore,  as 
seen  in  Figure  4-31,  one  obtains  the  following:  In  the  first  sub-bin  (0<ESR<375),  SA  has  an  average  value  of  0.32 
and  a  standard  deviation  of  0.23.  In  the  second  sub-bin  (375<ESR<750),  SA  has  an  average  value  of  0.40  and  a 
standard  deviation  of  0.28.  In  the  third  sub-bin  (750<ESR<1 130),  SA  attains  an  average  value  of  0.52  and  a 
standard  deviation  of  0.20.  Finally,  in  the  fourth  sub-bin  (1 130<ESR<  15(H)),  SA  has  an  average  value  of  0.56  and  a 
standard  deviation  of  0. 19. 


Figure  4-32  shows  the  variation  of  SA  as  a  function  of  range.  S  A  attains  its  highest  value  of  0.897  at  5370  and  Ltien 
falls  to  around  0.7  at  around  7000. 

4.6  TBL's  PROBABILITY  OF  POST-SEGMENT  DETECTION  VS.  METRICS  ON  THE  COMBINED  TI 
AND  29-PALM  DATA  BASE 

4.6.1  PD  vs.  TIR2 

Figure  4-33  shows  the  variation  of  Pq  as  a  function  of  TIR2.  As  TIR2  increases  Pp>  tends  to  increase  and  attain  the 
value  of  1  for  most  TIR2  values  higher  than  350. 

4.6.2  PDvs.TBIR2 

Figure  4-34  shows  the  variation  of  Pq  as  a  function  of  TBIR2.  As  TBIR2  increases  Pg>  lends  to  increase.  In  the 
ranges  of  (0<TBIR2<10)  the  Pq  average  is  0.70  with  a  standard  deviation  of  0  21.  For  (10<TBIR2<20),  Pg>  has  an 
average  value  of  0.56  and  a  standard  deviation  of  0.35.  For  (20<TBfR2<30),  PD  has  an  average  value  of  0.53  and  a 

standard  deviation  of  0.50.  For  (30<TBIr2<1  10),  Pq  has  an  average  value  of  0.5  and  a  standard  deviation  of 
0.577. 

4.6.3  PDvs.  ESR 

Figure  4-35  shows  the  variation  of  Pp>  as  a  function  of  ESR.  For  values  of  (0<ESR<50),  the  average  value  of  Pp 
was  0.61  and  the  standard  deviation  of  0.32.  For  values  of  (50<ESR<100),  the  average  value  of  PD  was  0.91  and  a 
standard  deviation  was  0. 12.  For  (100<ESR<1500),  the  average  value  of  Pg>  was  0.67  with  the  standard  deviation 
of  0.4.  For  values  of  (500<ESR<7200),  the  average  Pq  was  0.6  and  the  standard  deviation  was  0.459 

4.6.4  PD  vs.  Range 

Figure  4-36  shows  the  variation  of  Pq  as  a  function  of  range.  Worst  Pp>s  occur  at  the  9800  range.  P^  has  the 
maximum  value  of  0.53  at  ranges  between  1400  to  2100,  Pp  has  a  maximum  of  1  and  a  minimum  of  around  0.60. 

4.6.5  P[)  vs.  False  Alarm  Range  for  TBL 

Figure  4-37  shows  the  variation  of  Pp  per  frame  as  a  function  of  false  alarm  rate  (FAR)  performance.  The  numbers 
inside  the  parentheses  arc  the  numbers  of  samples  having  the  same  position  on  the  graph  As  can  be  seen,  a  huge 


number  of  samples  (53)  have  their  Pp  equal  to  1  A  total  ol  54  samples  have  P^  saiues  around  o  rr  1  he  rest  <>l 
the  55  samples  have  Pp>  values  of  0.5  or  smaller. 

Figure  4-38  shows  the  variations  of  Pq  per  frame  range  coded:  each  sample  is  specified  as  to  its  range  value  For 
Pp)>0.90  and  FAR<5,  the  samples  of  short  range  625  to  1250  dominate.  The  following  list  shows  the  range  values 
and  their  corresponding  code  name: 


A  =  625  -  1250 

B  =  1251  -  1875 

C  =  1876-  1875 

D  =  2501  -3125 

E  =  3126-  3750 


F  =  3751  -4375 

G  =  4376  -  5000 

H  =  5001  -  5625 

I  =  5626  -  6250 

=  6251  -6875 


K  =  6875  -  7500 

L  =  7501-  8125 

M  =  8126-  8750 

=  8751  -9375 

=  9375  -  10,000 


J 


N 
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Figure  4-38.  Variation  of  Pq  per  Frame  as  a  Function  of  FAR  per  Frame, 
Range  Coded,  on  the  Combined  TI  and  29-Palm  Data  Base 


SECTION  5 


COMPARISON  OFTBL  AND  BACORE 

The  objective  of  this  task  was  to  compare  the  performances  of  the  BACORE  and  TBL  on  the  same  data  base  (29- 
palm  and  TI)  and  the  same  number  of  frames.  In  our  comparison,  163  frames  for  both  BACORE  and  for  TBL  were 
used.  The  division  of  frames  was:  135  frames  of  TI  data  for  each  case,  and  28  frames  29-palm  data  for  each  case. 

Table  5-1  shows  the  results  of  the  comparison  of  the  average  segmentation  accuracy,  probability  of  post  segment 
detection,  and  false  alarm  rate  per  frame  for  the  two  algorithms.  TBL  performance  in  almost  every  aspect  lags  the 
BACORE  performance.  Table  5-2  shows  the  results  for  the  TI  data  base  and  Table  5-3  show  the  29-palm  results 

Table  5-2  shows  that  BACORE  has  higher  SA,  higher  Pp.  However,  for  this  data  base  FAR  was  lower  for  TBL 
than  BACORE. 

Table  5-3  shows  that  segmentation  accuracy  for  BACORE  was  lower  than  the  corresponding  value  for  the  TBL  on 
the  29-palm  data  base.  PD  values  for  the  two  algorithms  are  similar.  However,  FAR  was  extremely  higher  for  TBL 
(almost  6  times)  than  the  corresponding  value  for  BACORE  at  the  output  of  the  clutter  rejection. 

Overall  BACORE  outperforms  TBL,  even  though  the  strength  of  BACORE  is  in  multi-frame  processing  and  not  in 
single-frame  processing. 

Table  5-4  shows  a  comparison  of  TBL,  BACORE  single-frame,  and  BACORE  multi-frame  processors.  The 
BACORE  multi-frame  processor  is  seen  to  outperform  to  a  much  higher  degree  the  performances  of  TBL  and  single 
frame  BACORE,  especially  in  the  area  of  FAR.  It  should  be  noted,  however,  that  the  data  base  used  for  multi¬ 
frame  BACORE  was  different  from  those  of  the  others. 


TABLE  5-3.  COMPARISON  OF  TBL  AND  BACORE  ON  THE  29-PALM  DATA  BASE 
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TABLE  5-4.  COMPARISON  OF  TBL,  BACORE  SINGLE-FRAME,  AND  B ACORE  MULTI-FRAME 


SECTION  6 


TEST  EXPERIMENTS  FOR  BACORE  SEGMENTATION 
AND  DETECTION  ALGORITHMS 


As  an  important  part  of  the  PA1RSTECH  Project,  we  used  two  data  bases  that  were  supplied  by  ERIM  (see  Section 
2.3)  to  test  the  detection  and  segmentation  modules  of  BACORE 

6.1  DETECTION  TEST  EXPERIMENTS 

We  ran  our  BACORE  detector  on  a  total  of  200  frames  of  unground  iruthcd  images.  We  mailed  the  detected 
regions  of  interest  (ROI. s')  to  ERIM  as  required. 

Honeywell's  MTAP  is  an  integrated  system  that,  when  broken  into  isolated  parts,  may  produce  suboptimum  results. 
In  our  integrated  system,  for  the  purpose  of  having  high  PD,  we  allow  a  large  number  of  false  alarms  dial  will  be 
dealt  with  at  die  intermediate  level  and  higher  levels.  However,  this  fact  is  not  transparent  at  die  end  of  the 
detection  modules.  Consequently,  we  anticipate  that,  for  our  detection  algorithm,  FAR  will  be  relatively  wry  high. 

The  detection  experiment  was  done  at  five  different  ranges.  A  total  of  1000  runs  were  made. 

6.2  SEGMENTATION  TEST  EXPERIMENT 

ERIM  supplied  us  with  four  or  five  different  ROl  detection  seeds  per  frame  to  be  used  on  200  frames  of  unground 
truthed  data  The  experiment  was  done  at  five  different  ranges  for  each  frame  This  experiment  presented  some 
difficulties  for  the  BACORE  segmentor.  This  is  an  integrated  system;  the  result  of  separating  the  segmentor  from 
the  detector  was  unpredictable  and  potentially  could  lead  to  nonoptimum  performance  results.  For  example,  the 
BACORE  uses  the  ROI  region,  obtained  from  die  detector,  at  the  last  stage  of  the  segmentation  to  separate  clutter 
from  the  true  target  regions.  This  is  done  by  keeping  only  those  regions  whose  overlap  area  with  the  R(  )1  is  not 
zero.  Since,  lor  the  lest  experiment,  the  ROI  consists  of  only  one  single  point,  the  output  of  the  segmentation,  in 
many  eases,  tended  to  be  zero,  simply  because  of  the  htrge  possibility  that  exists  for  the  seed  to  be  nonoverlapping 
with  the  segmented  region.  es|vcially  in  himodal  targets.  Removing  the  last  stage  of  the  segmentation,  however, 
can  potentially  lead  to  die  admittance  of  some  clutter  regions.  We  used  this  last  step  occasionally  m  our  test 
experiment.  The  segmented  images  were  pul  in  proper  formats  and  mailed  to  ERIM  lor  their  evaluation 

The  results  ol  the  evaluation  of  the  detection  and  segmentation  test  experiment  are  not  available  set  and  wilt  Iv 
published  by  ERIM  at  a  later  time. 
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SECTION  7 


CONCLUSION  AND  SUMMARY  OF  RESULTS 

In  this  study  we  investigated  the  detection  and  segmentation  performances  of  Honeywell  BACORE  and  TBL 
algorithms.  The  measures  of  performance  were  probability  of  detection  (Pj^),  false-alarm  rate  (FAR)  ,  and 
segmentation  accuracy  (SA). 


As  part  of  this  study  we  characterized  and  ground  truthed  six  sequences  of  multi-frame  sequential  data.  The  image 
characterization  was  in  the  form  of  evaluating  them  in  terms  of  several  image  metrics,  among  them  the  TIR^, 
TBIR^  and  ESR.  ERIM  supplied  us  with  ground  truthed  and  similarly  characterized  single  frame  data. 

All  of  the  performance  measures  were  tabulated  in  terms  of  the  image  metrics  for  a  total  of  163  frames  and  448 
targets  for  the  single-frame  evaluations.  For  the  multi-frame  evaluations  a  total  of  120  frames  containing  700 
targets  were  used.  For  BACORE  multi-frame,  seven  models  were  generated  relating  the  performance  measures  to 
the  TIR^,  TBIR2,  ESR  and  range  metrics.  These  models  are  to  be  used  in  performance  prediction. 


The  results  of  the  study  show  that  the  performance  of  both  BACORE  and  TBL  on  single-frame  data  is  relatively 
similar,  with  BACORE  performing  mostly  better  than  TBL.  Segmentation  accuracy  values  of  both  TBL  and 
BACORE  single-frame  were  relatively  low,  smaller  than  0.4.  This  was  partly  due  to  the  inadequate  way  the 
ATRWG  measures  of  segmentation  performance  are  defined  and  partly  due  to  the  way  the  ground  truth  data  were 
prepared  by  ERIM.  In  the  ERIM  ground  truth  data  base,  targets  were  represented  by  a  wire  frame  model  with  no 
consideration  paid  to  their  thermal  characteristics. 


The  study  showed  that  there  were  not  significant  correlations  between  TIR^,  TBIR^,  and  ESR  and  the  performance 
measures  of  BACORE  and  TBL.  Moreover,  most  of  the  imagery  were  clustered  around  low  values  of  TIR^,  and 
TBIR^  and  this  made  the  study  meaningful  only  for  a  small  region  of  metric  space. 


The  study  showed  that  generally,  as  TIR^,  TBIR^  and  ESR  increase,  at  least  in  t  he  lower  domains  of  metric  space, 
performance  measures  increase.  However,  for  higher  domains  of  metric  space,  the  behavior  of  algorithms  was 
different.  This  is  partly  due  to  the  lack  of  sufficient  data  in  this  domain  and  consequently  relying  on  few  samples. 


The  study  of  multi-frame  BACORE  showed  a  marked  improvement  in  performance  over  single-frame  BACORE. 
This  study  also  showed  the  significance  of  the  clutter  rejection  module  in  the  BACORE  algorithm  for  reducing 
FAR.  On  every  data  base  that  we  used,  clutter  rejection  reduced  FAR  by  almost  four  to  six  times.  However,  this 
was  accompanied  by  reduction  in  the  values  of  the  detection  probabilities. 
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As  part  of  this  study,  we  gained  significant  insight  into  the  BACORE  and  TBL  algorithm  parameter  selection. 

For  BACORE,  the  number  of  parameters  that  had  significance  on  performance  measures  found  to  be  nine  out  of  a 
total  of  18.  For  TBL,  the  three  parameters  were  found  to  be  functions  of  the  image  metrics  and  were  set  to  provide 
optimum  quantitative  performance. 

Setting  of  the  parameters  of  algorithms  was  found  to  be  crucial  in  the  overall  performance  of  both  TBL  and 
BACORE.  In  particular,  our  sensitivity  studies  of  TBL  showed  that,  in  any  cases,  performance  measures  are  very 
sensitive  to  the  values  of  the  algorithm  parameters. 
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